Notifications framework for distributed software upgrades

ABSTRACT

Techniques for managing an upgrade operation comprising multiple upgrade process executing on multiple host machines (or hosts) for upgrading software applications on the multiple hosts. Techniques are disclosed for managing notifications that are generated by the multiple upgrade processes during execution, and more particular, techniques for reducing the number of notifications that are sent to a user. The techniques include: only sending a subset of the generated notifications to a user, the subset being selected at the host machines based upon notifications level criteria specified by the user for the host machines; consolidating multiple generated notifications into a fewer number of consolidated notifications and only sending consolidated notifications to the user; combination of criteria-based selection and notifications consolidation.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/935,254, filed Nov. 6, 2015 (now allowed), which claims priority andbenefit from U.S. Provisional Patent Application No. 62/076,900, filedNov. 7, 2014 and entitled “NOTIFICATIONS FRAMEWORK FOR DISTRIBUTEDSOFTWARE UPGRADES.” The entire contents of the 14/935,254 and 62/076,900applications are incorporated herein by reference for all purposes.

BACKGROUND

The term upgrading a software application generally refers to theprocess of replacing an existing version of the software applicationwith a newer version, adding a new version of the software applicationwhere none previously existed, or somehow changing an existing versionof the software application to a newer different version. A softwareupgrade may be performed for various reasons such as to add one or morefeatures, remove one or more features, modify one or more features froman existing version of the software, remove bugs or errors, improve thesoftware efficiency, and other reasons. An upgrade is generallyperformed to enhance the performance of a software application.

Many modern computing environments typically include a framework ofmultiple heterogeneous software applications, which may be developed bydifferent third-party entities. These software applications may includezero or more plugins. The plugins may, for example, include softwarecomponents that add a new utility/feature or enhance theutilities/features of a software application. The applications may runon or be hosted by multiple hosts in a distributed environment, witheach host potentially hosting multiple applications. Performing anupgrade operation in such a distributed environment comprises multipleupgrade processes executed by multiple hosts such that one or moreupgrade processes may be executed to upgrade each application. Many ofthe upgrade processes may be executed in parallel. Possible dependenciesamong the upgrade processes further complicate the overall upgrade.

In an environment where multiple applications hosted by multiple hostsneed to be upgraded, the upgrade operation can comprise several upgradeprocesses running, possibly in parallel, on multiple hosts. Each upgradeprocess may be configured to generate and send out a message ornotification indicative of the progress of the process. Thesenotifications may, for example, be provided in the form of emails thatare sent to one or more users such as system administrators and thelike. Since a single host may execute multiple upgrade processes,possibly in parallel, with each upgrade process generating one or morenotifications, a host may generate and send out multiple suchnotifications. Since there could be multiple hosts (e.g., thousands ofhosts) involved in the upgrade operation with each host executing one ormore upgrade processes, a large number of notifications may be generatedas part of the overall upgrade operation. Significant processing andmemory resources are used, and many times wasted, in the generation anddelivery of these large numbers of notifications. Moreover, the usersreceiving these notifications are completely overwhelmed by the volumeof such notifications.

Conventionally, individual user recipients have tried to address thisproblem by building their own filtering applications to reducenotifications received by that specific recipient. However, thesefiltering applications are user-specific and execute on the user-side(on the user's client devices) and thus fail to reduce the overallnotifications-related network traffic generated by the hosts executingthe upgrade processes.

BRIEF SUMMARY

The present disclosure relates generally to managing an overall upgradeoperation comprising multiple upgrade process executing on multiple hostmachines (or hosts) for upgrading software applications on the multiplehosts. In certain embodiments, techniques are disclosed for managingnotifications that are generated by the multiple upgrade processesduring execution. For example, an upgrade process may generate one ormore notifications indicating the progress of the upgrade process whenthe notification is generated. The techniques reduce the number ofnotifications that are sent to a user. As a result, compared toconventional techniques, the number of network, processing, and memoryresources needed for processing of the notifications, both on the hostmachines side and also on the client devices of users that receive thenotifications, are dramatically reduced.

In certain embodiments, according to one technique, instead of sendingall the notifications generated by the upgrade processes to a user, onlya subset of the generated notifications are sent to a user, the subsetselected at the host machines based upon notifications level criteriaspecified by the user for the host machines. In one embodiment, for anotification generated by an upgrade process executing on a hostmachine, a level associated with the generated notification isdetermined and compared to a user-specified notification level for thathost machine. A determination is then made whether or not to send thenotification to the user based upon the comparison. The notification issent to the user only if the comparison indicates that the notificationis to be sent to the user, else the notification is not sent to the userand filtered out.

In certain embodiments, according to another technique, multiplenotifications generated by the upgrade processes may be consolidatedinto a few number of consolidated notifications (i.e., fewer than thenumber of multiple notifications generated by the upgrade process), andthe consolidated notifications sent to a user instead of the multiplegenerated notifications. For example, multiple notifications may beconsolidated into a single consolidated notification, which is then sentto the user instead of the multiple notifications.

In certain embodiments, combinations of the filtering and consolidationtechniques may be used to reduce the number of notifications that arecommunicated to users. As notifications are generated by the upgradeprocesses executing on a host, the level of each generated notificationis compared to the level(s) specified in the configuration fileconfigured for that host. The comparison determines whether thatnotification is to be communicated to a recipient or is not to becommunicated (i.e., is to be filtered out). In this manner, anotification filtering system is implemented on a host that onlycommunicates notifications with a certain level(s) to one or morerecipients and does not communicate the other notifications that arefiltered out. A recipient could a user, such as a system administrator,or a notification agent, or the like. In some embodiments, one of thehosts may be assigned as the notification agent. Alternatively, theupgrade system may include a dedicated notification agent.

In certain embodiments, techniques (including methods, systems, code orsoftware instructions executed by one or more processors) are providedfor generating, by an upgrade process executing on a host machine forupgrading a software application on the host machine, a notificationcomprising information related to the upgrade process. The host machinedetermines a notification level associated with the notification. Thehost machine determines, based upon the notification level associatedwith the notification and a user-specified notification level configuredfor the host machine, whether the notification is to be sent to a user.Determining whether the notification is to be sent to the user comprisescomparing the notification level associated with the notification to theuser-specified notification level, and determining that the notificationis to be sent to the user if the notification level associated with thenotification matches the user-specified notification level. Determiningwhether the notification is to be sent to the user comprises comparingthe notification level associated with the notification to theuser-specified notification level, and determining that the notificationis to be sent to the user if the notification level associated with thenotification matches or is higher than the user-specified notificationlevel. The host machines send the notification to the user upondetermining that the notification is to be sent to the user. The hostmachine does not send the notification to the user upon determining thatthe notification is to not be sent to the user. The host machine storesa configuration file, the configuration file storing the user-specifiednotification level. The notification comprises information indicating anupgrade progress status of the upgrade process. The notification is sentto a notification agent to be consolidated with additional notificationsprior to being sent to the user.

In certain embodiments, techniques (including methods, systems, code orsoftware instructions executed by one or more processors) are providedfor receiving, by a third host machine from a group of host machines, afirst notification generated by a first upgrade process executing on afirst host machine for upgrading a first software application on thefirst host machine. The first notification comprises information relatedto the first upgrade process. The first host machine is included in thegroup of host machine. The third host machine receives a secondnotification generated by a second upgrade process executing on a secondhost machine for upgrading a second software application on the secondhost machine. The second notification comprises information related tothe second upgrade process. The second host machine included in thegroup of host machines. The third host machine consolidates the firstnotification and the second notification into a consolidatednotification. The third host machine determines whether the consolidatednotification is to be sent to a user. The third host machine sends theconsolidated notification to the user, instead of the first notificationand the second notification, upon determining that the consolidatednotification is to be sent to the user. The third host machine does notsend the consolidated notification to the user upon determining that theconsolidated notification is not to be sent to the user. A thirdnotification may be generated by a third upgrade process executing onthe third host machine for upgrading a third software application on thethird host machine. The third notification comprises information relatedto the third upgrade process. The third host machine included in thegroup of host machines. The third host machine consolidates the thirdnotification with the first notification and the second notificationinto the consolidated notification.

In some embodiments, determining whether the consolidated notificationis to be sent to the user further comprises the third host machinedetermines a notification level associated with the consolidatednotification. The third host machine determines based upon thenotification level associated with the consolidated notification and auser-specified notification level configured for the group of hostmachines, whether the notification is to be sent to the user.

According to some embodiments, the third host machine compares thenotification level associated with the consolidated notification to theuser-specified notification level, and determines that the consolidatednotification is to be sent to the user if the notification levelassociated with the consolidated notification matches the user-specifiednotification level. In some embodiments, the third host machine comparesthe notification level associated with the consolidated notification tothe user-specified notification level, and determines that theconsolidated notification is to be sent to the user if the notificationlevel associated with the consolidated notification matches or is higherthan the user-specified notification level.

In some embodiments, the first host machine determines a firstnotification level associated with the first notification, and comparesthe first notification level associated with the first notification to afirst user-specified notification level configured for the first hostmachine. The first host machine then determines that the firstnotification is to be sent to the third host machine based on thecomparing. Yet in other embodiments, the third host machine determines afirst notification level associated with the first notification and asecond notification level associated with the second notification. Thethird host machine compares the first notification level and the secondnotification level to a user-specified notification level configured forthe group of host machines, and determines that the first notificationand the second notification are to be consolidated based on thecomparing.

In certain embodiments, techniques (including methods, systems, code orsoftware instructions executed by one or more processors) are providedfor generating, by an upgrade process executing on a host machine forupgrading a software application on the host machine, a notificationcomprising information related to the upgrade process. The host machineis provided in a group of host machines. The host machine determines anotification level associated with the notification. The host machinestores a configuration file, the configuration file storing auser-specified notification level for the host machine, and a groupconfiguration file, the group configuration file storing auser-specified group notification level for the group of host machines.The host machine determines, based upon the notification levelassociated with the notification, the user-specified notification level,and the user-specified group notification level, whether thenotification is to be sent to a user. The host machine sends thenotification to the user upon determining that the notification is to besent to the user. The host machine does not send the notification to theuser upon determining that the notification is to not be sent to theuser. The notification comprises information indicating an upgradeprogress status of the upgrade process. The notification is sent to anotification agent to be consolidated with additional notificationsprior to being sent to the user.

In some embodiments, the determining whether the notification is to besent to the user further comprises comparing, by the host machine, thenotification level associated with the notification to theuser-specified group notification level and the user-specifiednotification level. The user-specified group notification level trumpsthe user-specified notification level for the notification. The hostmachine determines that the notification is to be sent to the user ifthe notification level associated with the notification matches theuser-specified group notification level. In other embodiments, the hostmachine determines that the notification is to be sent to the user ifthe notification level associated with the notification matches or ishigher the user-specified group notification level.

In some embodiments, the determining whether the notification is to besent to the user further comprises comparing, by the host machine, thenotification level associated with the notification to theuser-specified group notification level and the user-specifiednotification level. The user-specified notification level trumps theuser-specified group notification level for the notification. The hostmachine determines that the notification is to be sent to the user ifthe notification level associated with the notification matches theuser-specified notification level. In other embodiments, the hostmachine determines that the notification is to be sent to the user ifthe notification level associated with the notification matches or ishigher the user-specified notification level.

These and other embodiments are described in further detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the present invention are described indetail below with reference to the following drawing figures:

FIG. 1 depicts an upgrade infrastructure for executing multiple upgradeprocesses on multiple hosts according to an embodiment of the presentinvention.

FIG. 2 depicts an upgrade infrastructure including multiple hostsgrouped into a pod for executing multiple upgrade processes according toan embodiment of the present invention.

FIG. 3 depicts a simplified flowchart depicting processing performed bythe upgrade infrastructure for filtering notifications by a host machinebefore sending the notifications to a user, according to an embodimentof the present invention.

FIG. 4 depicts a simplified flowchart depicting processing performed bythe upgrade infrastructure for consolidating notifications generated bymultiple upgrade processes hosted by multiple host machines beforesending the notifications to a user, according to an embodiment of thepresent invention.

FIG. 5 depicts a simplified diagram of a distributed system forimplementing one of the embodiments.

FIG. 6 depicts an exemplary computer system that may be used toimplement an embodiment of the present invention.

FIG. 7 depicts a simplified flowchart depicting a method performed by ahost machine for determining whether a notification is to be sent to auser according to an embodiment of the present invention.

FIGS. 8A and 8B together depict a simplified flowchart for determiningwhether a notification generated by a host machine is to be sent to auser when the host machine is part of a group of host machines (e.g., apod) according to an embodiment of the present invention.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofembodiments of the invention. However, it will be apparent that variousembodiments may be practiced without these specific details. The figuresand description are not intended to be restrictive.

Systems depicted in some of the figures may be provided in variousconfigurations. In some embodiments, the systems may be configured as adistributed system where one or more components of the system aredistributed across one or more networks.

The present disclosure relates generally to managing an overall upgradeoperation comprising multiple upgrade process executing on multiple hostmachines (or hosts) for upgrading software applications on the multiplehosts. The software applications hosted by the hosts may be developed bydifferent third-party entities. Due to the heterogeneity of the softwareapplications and further coupled with the distributed nature of thecomputing environment, the process of upgrading such applications isquite complicated. The upgrade is performed by running one or multiplesoftware upgrades on the various hosts for upgrading the various piecesof software applications hosted by the hosts. The complexity ofperforming such a distributed upgrade is impacted by several factorssuch as the number of applications that need to be upgraded, thedistributed nature of the applications, potential upgrade dependenciesbetween upgrade processes for the applications hosted by the same ordifferent hosts, customized upgrade requirements for individualapplications, the number of hosts involved in the upgrade, and otherfactor.

An upgrade infrastructure, as described herein, enables softwareupgrades to be performed in a heterogeneous and distributed computingenvironment in an automated and efficient manner. The upgradeinfrastructure enables two or more upgrade processes hosted by the samehost or hosted by multiple different hosts to synchronize and coordinatetheir upgrade activities in an automated manner. In certain embodiments,the upgrade infrastructure provides multi-level consolidatednotifications to inform users of the upgrade progress status. Eachupgrade process may generate notifications. The hosts may associatelevels with the generated notifications according to a pre-determinedcriteria. The levels may correspond to different severity or importancelevels in a notification hierarchy.

In certain embodiments, techniques are disclosed for managingnotifications that are generated by the multiple upgrade processesduring execution. For example, an upgrade process may generate one ormore notifications indicating the progress of the upgrade process whenthe notification is generated. The techniques reduce the number ofnotifications that are sent to a user. As a result, compared toconventional techniques, the number of network, processing, and memoryresources needed for processing of the notifications, both on the hostmachines side and also on the client devices of users that receive thenotifications, are dramatically reduced.

In certain embodiments, according to one technique, instead of sendingall the notifications generated by the upgrade processes to a user, onlya subset of the generated notifications are sent to a user, the subsetselected at the host machines based upon notifications level criteriaspecified by the user for the host machines. In one embodiment, for anotification generated by an upgrade process executing on a hostmachine, a level associated with the generated notification isdetermined and compared to a user-specified notification level for thathost machine. A determination is then made whether or not to send thenotification to the user based upon the comparison. The notification issent to the user only is the comparison indicates that the notificationis to be sent to the user, else the notification is not sent to the userand filtered out.

In certain embodiments, according to another technique, multiplenotifications generated by the upgrade processes may be consolidatedinto a few number of consolidated notifications (i.e., fewer than thenumber of multiple notifications generated by the upgrade process), andthe consolidated notifications sent to a user instead of the multiplegenerated notifications. For example, multiple notifications may beconsolidated into a single consolidated notification, which is then sentto the user instead of the multiple notifications.

In certain embodiments, combinations of the filtering and consolidationtechniques may be used to reduce the number of notifications that arecommunicated to users.

FIG. 1 illustrates an exemplary upgrade infrastructure (e.g. upgradesystem) 100 according to various embodiments. The upgrade infrastructure100 includes multiple hosts 110, 120, 130. The hosts 110, 120, 130 mayhost multiple software applications. When these software applicationsare to be upgraded, multiple upgrade processes 112, 114, 122, 124, 132,134 may be executed by the hosts 110, 120, 130 to upgrade the softwareapplications hosted by the hosts 110, 120, 130.

In the embodiment depicted in FIG. 1, the upgrade infrastructure 100includes an upgrade console 104 in communication with the hosts 110,120, 130 through a communication network 108. The upgrade console 104may be a computing system that a user 102 (e.g. a system administrator)may interact with to initiate and control the overall upgrade on thehosts 110, 120, 130. The upgrade console 104 may include an upgradeorchestrator 106 for initiating, coordinating and synchronizing theupgrade processes 112, 114, 122, 124, 132, 134 hosted by the hosts 110,120, 130. Status information related to the overall upgrade may beoutput to the user via upgrade console 104.

Various embodiments may enable the user 102 to directly interact withthe hosts 110, 120, 130 through the communication network 108 toinitiate and control the overall upgrade on the hosts 110, 120, 130. Insuch embodiments, the user 102 may not use the upgrade console 104 tointeract with the hosts 110, 120, 130.

In certain embodiments, the upgrade may be performed in phases. Duringeach phase, multiple upgrade processes (i.e. tasks) may be hosted by thehosts 110, 120, 130. The upgrade orchestrator 106 may ensure that acurrent set of upgrade processes run to successful completion on allhosts before proceeding with the next set of upgrade processes. One ofordinary skill in the art will appreciate that the upgradeinfrastructure 100 may include any number of components, hosts andupgrade processes. Thus, the upgrade infrastructure 100 is not limitedto the components, hosts and upgrade processes illustrated in FIG. 1.

As illustrated in FIG. 1, multiple upgrade processes may be hosted byeach one of the hosts 110, 120, 130. For example, upgrade processes 112,114 may be hosted by on host 110, upgrade processes 122, 124 may behosted by on host 120 and upgrade processes 132, 134 may be hosted by onhost 130. Two or more of the upgrade processes 112, 114, 122, 124, 132,134 may run in parallel. In various embodiments, the upgrade processes112, 114, 122, 124, 132, 134 may be serialized. The upgrade orchestrator106 may synchronize two or more of the upgrade processes 112, 114, 122,124, 132, 134 using a dedicated upgrade orchestrator module 116, 126,136 executed on each host 110, 120, 130, respectively. Each upgradeorchestrator module 116, 126, 136 may be a piece of code be hosted bythe host 110, 120, 130 for performing the upgrade activities. Theupgrade orchestrator modules 116, 126, 136 may initiate and control theupgrade processes 112, 114, 122, 124, 132, 134 executed on the hosts110, 120, 130. The upgrade orchestrator modules 116, 126, 136 mayreceive and/or respond to commands and instructions from the upgradeconsole 104 to facilitate the upgrade on the hosts 110, 120, 130.

The upgrade processes 112, 114, 122, 124, 132, 134 executed on the hosts110, 120, 130 may generate notifications. A notification generated by anupgrade process may include information related to the status of theupgrade process when the notification is generated. An upgrade processmay generate one or more notifications, potentially at different timepoints during its execution on a host. For example, a notificationgenerated by upgrade process 112 may include information indicative of astatus of upgrade process 112, a notification generated by upgradeprocess 114 may include information indicative of a status of upgradeprocess 114, and so on. Upgrade process 112 may generate notificationsat different time points during its execution, with a notificationgenerated at a particular time point including information about thestatus of up 112 at that time point.

According to certain embodiments, the notifications generated by theupgrade processes are associated with levels (e.g., severity orimportance levels) according to a pre-determined criteria. For example,the notifications may be associated with an indicator (e.g., a flag, atext field) indicating the level associated with the notification. Forexample, in one embodiment, three notification levels may be predefined:an INFO level, an ALERT level, and a PAUSEPOINT level. Each generatednotification may be associated with one of these three levels byassociating a flag or a text field with the notification indicating thespecific level. For example, the notifications associated with the INFOlevel may include status change notifications generated by one or moreof the upgrade processes 112, 114, 122, 124, 132, 134. The notificationsassociated with the ALERT level may include upgrade failurenotifications generated by one or more of the upgrade processes 112,114, 122, 124, 132, 134. The notifications associated with thePAUSEPOINT level may indicate that one or more of the upgrade processes112, 114, 122, 124, 132, 134 are paused.

In certain embodiments, two or more of the levels may be organized in ahierarchy or ordering (e.g. a severity or an importance hierarchy). Forexample, a hierarchy may be defined for the PAUSEPOINT, ALERT, and INFOlevels discussed above such that PAUSEPOINT corresponds to a higherhierarchical level than the ALERT level, and the ALERT level correspondsto a higher hierarchical level than the INFO level. As discussed below,the hierarchical relationships between the levels may influence thenotifications that are sent to the users.

A user 102 in communication with the hosts 110, 120, 130 may define,either through direct interaction with the hosts 110, 120, 130 orthrough the communication network 108, one or more configuration files119, 129, 139 for the hosts 110, 120, 130. The configuration files 119,129, 139 may enable the user 102 to indicate the user's preference toreceive notifications corresponding to a given level. For example, theuser 102 may wish to receive notifications associated with a particularlevel (or levels), e.g. the ALERT level. In some embodiments,configuration file may be defined at a host-level, and include userpreferences for multiple users. For example, configuration file 119, 129or 139 may include notification preferences of two or more users. Afirst user (e.g., user_1) may be an upgrade administrator and may wishto receive ALERT level notifications (e.g., notifications for criticalevents such as a patching failure). A second user (e.g., user_2) maywish to receive all notifications generated by the host. The userpreference for user_1 and user_2 may be stored in the same configurationfile, (e.g., the configuration file for the host).

The notification filters 118, 128, 138 may use the configuration files119, 129, 139 to filter out any notifications that are not associatedwith the user-specified level. Any notifications associated with levelsthat do not match the user-specified level (in the configuration file)are not communicated to that user, resulting in those notificationsbeing filtered out for that particular user. The hosts 110, 120, 130 maysend the remaining notifications (i.e. notifications associated with theuser-specified level) to the user 102.

As described above, in certain embodiments, the different notificationlevels may be hierarchically related to each other. The user 102 mayspecify such a hierarchical level for a host and information indicativeof the hierarchical level may be stored in the configuration file forthat host. In one such embodiment, when the user 102 specified a levelfor receiving notifications, the upgrade infrastructure 100 may not onlyenable notifications matching the user-specified level to becommunicated to the user 102 but also allow notifications at higherhierarchical levels (e.g. more severe or more important notifications)than the user-specified hierarchical level to be sent to the user 102 aswell. In such an embodiment, the user-specified hierarchical level maybe considered a lower bound for notifications to be sent to the user.

For example, in an embodiment where the PAUSEPOINT, ALERT, and INFOlevels are hierarchically related, with PAUSEPOINT being higher (or moresevere/important) than ALERT, and ALERT being higher (or moresever/important) that INFO, if the user 102 indicates for a host thatthe user 102 wishes to receive notifications associated with the ALERTlevel, the hosts may send notifications associated with the PAUSEPOINTlevel (e.g. a higher hierarchical level than the ALERT level) along withthe notifications associated with the ALERT level to the user. In thisexemplary embodiment, the notification filters 118, 128, 138 may filterout notifications associated with the INFO level (e.g. a lowerhierarchical level than the ALERT level). Accordingly, the user 102 mayreceive notifications that indicate an upgrade failure (e.g.notifications associated with the ALERT level) or notifications thatindicate paused upgrade processes (e.g. notifications associated withthe PAUSEPOINT level). The user 102 will not receive the less importantnotifications that indicate, for example, mere status changes (e.g.notifications associated with the INFO level).

According to various embodiments, different configuration files 119,129, 139 may be defined for different types of hosts 110, 120, 130. Thelevel (or levels) of interest specified by a user for one host may bedifferent from the level (or levels) specified by the same user for adifferent host. Various different factors may influence the level (orlevels) that the user specifies as of interest for a particular host,including but restricted to, the one or more applications hosted by thehost, the configuration of the host (e.g., networking, processing (e.g.,number of processors, speed of the processors, etc.), and memoryresources (e.g., system memory (e.g., RAM) resources of the host)), andthe like. For example, the user 102 may choose to receive only thenotifications associated with a first specified level from host 110while the user 102 may choose to receive notifications associated with asecond level from host 120, where the second level is different from thefirst level. The configuration files 119 and 129 may be set upaccordingly.

In certain embodiments, one or more hosts may be grouped together. Anexemplary group of hosts may be a pod. The term pod, as used herein,defines a modular set of resources including, for example, a specificset of infrastructure, middleware, and/or application resources. A podmay comprise one or more hosts. In certain embodiments, in a group ofhosts comprising multiple hosts, notifications generated by differenthosts within the group may be combined or consolidated into a fewernumber of notifications. The notification consolidation within a pod(e.g., an exemplary group of hosts) is discussed below with respect toFIG. 2. In general, consolidation of notifications can be performed inany environment where multiple hosts are logically grouped together andis not limited to pod environments.

FIG. 2 illustrates an exemplary upgrade infrastructure (e.g. upgradesystem) 200 according to various embodiments. The upgrade infrastructure200 includes similar elements to the upgrade infrastructure 100illustrated in FIG. 1. The reference numerals for these elements arekept the same and a discussion of these elements is provided above.Accordingly, the discussion of the similar elements is omitted here.

As illustrated in FIG. 2, the upgrade infrastructure 200 includesmultiple hosts 110, 120, 130. In the embodiment depicted in FIG. 2,hosts 110 and 120 are grouped together to form a pod 202. This is notmeant to be restrictive. In alternative embodiments, a pod may comprisevarious other combinations and numbers of hosts. Further, while only onepod 202 is depicted in FIG. 2, in alternative embodiments, the upgradeinfrastructure 200 may include additional pods formed by grouping two ormore hosts together. The upgrade infrastructure 200 may also include oneor more hosts (e.g. host 130) that are not part of a pod. In thismanner, upgrade infrastructure 200 may comprise some groupings of hostsand other non-grouped hosts.

As described above for FIG. 1, a user 102 may define one or moreconfiguration files 119, 129, and 139 for the individual hosts 110, 120,and 139, respectively to specify notifications that are of interest tothe user 102. Additionally, the user 102 may specify a configurationfile 209 for pod 202. Configuration file 209 specifies the notificationlevel(s) of interest to the user at the pod level. For example, the user102 may choose to receive only notifications associated with aparticular level from all hosts in the pod 202. Accordingly, the user102 may define the pod-level configuration file 209 to store informationindicative of this particular level. By configuring a configuration fileat the pod level, only the notifications associated with that particularlevel (or higher in the case of an hierarchical level) will be sent fromall hosts 110, 120 within the pod 202.

The configuration file for a pod level is independent from theconfiguration files specified for the individual hosts within that pod.In some embodiments, if a configuration file is defined at the podlevel, the user may not even define configuration files for one or moreindividual hosts included in that pod. For example, in the embodimentdepicted in FIG. 2, the user 102 may not define the host-levelconfiguration files 119 and 129 for hosts 110 and 120 respectively.

As described above, in some embodiments, in addition to the pod-levelconfiguration file that identifies a user's specified notification alertlevel(s) for the pod, a configuration file may be defined for one ormore hosts within that pod. In such a scenario, for a host for which ahost-specific configuration file is defined, the level specified in thehost-specific configuration overrides the level specified in thepod-level configuration file. For example, the user 102 may choose tospecify the ALERT level as the level of interest for the user for allhosts in the pod 202 except a selected subset of hosts, e.g. InternetDownload Manager (IDM) hosts. That is, even within a pod, the user canspecify different notification levels for different groups of hostswithin the pod. The user 102 may prefer to receive all notifications (asopposed to just the notifications associated with the user-specifiedALERT level) from the selected subset of hosts, e.g. the IDM hosts, inorder to have a close watch on the upgrade progress running on thesehosts. If host 120 illustrated in FIG. 2 is an IDM host, the user maydefine the host-level configuration file 129 such that all notificationsfrom host 120 will be sent to the user 102. In this exemplaryconfiguration, host 120 may send all notifications to user 102 whilehost 110 may only send the notifications associated with theuser-specified ALERT level.

In certain embodiments, upgrade processes 112, 114, 122, 124 to beexecuted on different hosts 110, 120 within a pod 202 may be started atapproximately the same time. Each upgrade process may be configured togenerate a starting notification upon starting execution. Accordingly,if there are 20 hosts in a pod, 20 starting notifications may be sent tothe user at approximately the same time. Moreover, every time an upgradeprocess restarts, the starting notification for the host executing theupgrade process may be resent. For an upgrade operation running on alarge number of hosts (e.g., thousands of hosts) a very large number(e.g., tens of thousands) of notifications (e.g., email notifications)may be sent just for start, restart, etc. In certain embodiments, theupgrade infrastructure 200 addresses this problem by consolidatingnotifications from multiple hosts resulting in a fewer notifications.Thus, instead of having each host send notifications individually, theupgrade infrastructure 200 may consolidate notifications into fewernotifications. In certain embodiments, the multiple notifications acrossmultiple hosts may be consolidated into one notification.

For purposes of explanation, it is assumed that consolidation is to beperformed for hosts within pod 202. In such an embodiment, one of thehosts within the pod may be identified as a notification agent for thehosts within the pod and only the notification agent is configured tosend out notifications for that pod. In one embodiment, the upgradeorchestrator 106 may identify one of the hosts in a pod as thenotification agent. In alternative embodiments, a dedicated notificationagent may be provided as part of upgrade infrastructure 100. Forexample, upgrade orchestrator 106 may select the host 110 as thenotification agent for the pod 202. Accordingly, only the host 110 maysend notifications for the pod 202.

According to various embodiments, a notification agent may send thenotifications in separate threads to prevent the notifications fromblocking the process of the overall upgrade. The notification agent maysend the notifications in multiple threads where each thread requiresless processing resources for transmitting the tread when compared tothe processing resources required for transmitting a single large threadincluding all notifications. Assigning a host as the notification agentmay be a static assignment (i.e. one host is selected to be anotification agent throughout the execution of the upgrade processes) ora dynamic assignment (i.e. different hosts become the notification agentthroughout the execution of the upgrade processes).

In certain embodiments, the selection of the notification agent fromamong a group of hosts may depend on the types of notificationsgenerated by the hosts. For example, a host may be selected as thenotification agent based on the host's ability to send startnotifications (i.e. notifications indicating that the upgrade processesbe hosted by the host have started execution). Which host operates asthe notification agent for sending start notifications may not be astatic (i.e. fixed) agent due to the fact that any single host may notalways run or may not always start before other hosts. Accordingly,which host operates as the notification agent for sending the startnotifications may be dynamic and thus, may vary over a period of time.For example, the upgrade orchestrator 106 may chose a first host frommultiple hosts that is the earliest one to start starts an upgradeprogress as the notification agent to send start notifications.

In some embodiments, the selected host may wait for other hosts to startexecuting upgrade processes and consolidate the start notifications fromall other hosts into a single start notification. The startnotifications generated by the upgrade processes starting execution onthe multiple hosts are all routed to the host that is the notificationagent. The notification agent host may then consolidate the multiplestart notifications received from the other hosts and also startnotifications generated by upgrade processes executed on thenotification agent host into a smaller number of notifications. Forexample, the multiple start notifications may be consolidated into asingle stated notification that indicates that multiple upgradeprocesses on the multiple hosts of a given pod have startedsuccessfully. In some embodiments, the notification agent may discardrestart notifications from any hosts within the pod.

In some embodiments, in order to allow multiple upgrade processes togenerate start notifications and for the notification agent host toreceive these start notifications, the consolidated start notificationmay be sent by the notification agent after a predefined delay after thetime the notification agent host was started successfully. Thepredefined delay may be configurable using a property in theconfiguration file 209 and may be configured such that reasonablelatency is allowed for hosts starting in a batch and sufficient time isallowed for the notification agent host to receive start notificationsfrom these multiple hosts. For example, the predefined delay may beconfigurable using EMAIL_STARTNOTIFICATION_DELAY defined inpod.properties configuration file. For example,EMAIL_STARTNOTIFICATION_DELAY=120, may indicate that the startnotification will be sent 120 seconds after the time the notificationagent host (e.g., the first host) was started successfully.

In certain embodiments, the upgrade infrastructure may provide anapplication programming interface (API) for the users to send customizednotifications without the need to code mail transport details. Thesupported transports may include simple message transfer protocol (SMTP)message transfer agent (MTA), mail submission agent (MSA) andauthenticated MSA connections. The supported transports may be run onall platforms. The overall upgrade operation and the user may be indifferent platforms with different infrastructures including differentoperating systems, using different e-mail relays, protocols, ports, etc.For all the platforms, the API may set secured or non-securedconnections to the e-mail relay including detecting the connection port,protocol, authentication, etc. and find recipients defined in theconfiguration file. The API may prepare the notification message byuploading/attaching any given attachments to the notification message,format the notification message, and send the notification message tothe e-mail relay. The API may also retry in case of failure. Thebuilt-in API significantly reduces the development effort by individualdevelopers by hiding the mail transport details.

The techniques discussed herein significantly reduce the notificationtraffic in a high scalable upgrade system through the use ofconfigurable filters, simultaneous notifications and consolidatednotifications across multiple hosts. The tens of thousands ofnotifications (e.g., start/stop notifications) that were previously sentfor a given upgrade process among the multiple upgrade processes of anoverall upgrade operation are reduced to one notification per pod thatincludes information for multiple hosts. In addition, tens of thousandsof conventional notifications may be automatically discarded based onuser-defined preferences requesting to receive only a selected set ofnotifications. Thus, techniques discussed herein dramatically reduce thenotifications-related traffic for that pod.

FIG. 3 depicts a simplified flowchart 300 depicting processing performedby the upgrade infrastructure for selectively choosing notificationsthat are sent to a user from a host machine from among multiplenotifications generated by upgrade processes executed by the hostmachine, according to an embodiment of the present invention. Theprocessing depicted in FIG. 3 may be implemented in software (e.g.,code, instructions, program) executed by one or more processing units(e.g., processors cores), in hardware, or using combinations thereof.The software may be stored on a non-transitory computer-readable medium(e.g., a memory device, a memory). The particular series of processingsteps depicted in FIG. 3 is not intended to be limiting. In oneembodiment, the processing depicted in FIG. 3 is performed by one ormore components of the upgrade infrastructure 100 depicted in FIG. 1.

The processing in flowchart 300 comprises two phases: a design timephase and a runtime phase. The design time phase corresponds to beforethe upgrade operation is started. The runtime phase corresponds to whenthe overall upgrade operation has been initiated. The description belowassumes that the host machines involved in the upgrade operation includea first host machine.

During the design time, at 302, information is stored on the first hostmachine identifying a user-specified level information. In certainembodiment, this information may be stored in a first configuration filestored on the first host machine. The first configuration file mayidentify a user-specified notification level that is then used by thefirst host machine to identify which notifications are to be sent to theuser and which notifications are to be not sent to the user (i.e.,filtered out). The user-specified notification level indicates anotification level that the user is interested in. In some embodiments,the user-specified level may be a hierarchical level.

In certain embodiments, the configuration file stored on a host machinemay store user-specified level preferences for multiple users. The levelpreference specified by a first user may be different from a levelpreference specified by a second user, and so on. In this manner, levelpreferences for multiple users may be stored in the configuration filestored on a host machine and the host machine may then use thisconfiguration information to determine which notifications to send toindividual users and which notifications to filter out. For example, thefirst configuration file may store user-specified level preferences forone or more users in addition to the first user.

The processing depicted in 304, 306, 308, 310, 312, and 314 is performedduring the runtime phase. At 304, as part of the overall upgradeoperation, a first upgrade process may be executed on the first hostmachine for upgrading a first software application on the first hostmachine. At 306, the first upgrade process may generate a firstnotification during execution on the first host machine. The firstnotification may include information related to the first upgradeprocess such as the status of the upgrade process as of the time thefirst notification is generated. For example, the first notification mayindicate that the first upgrade process has started execution, pausedexecution, about to finish execution, or that a failure occurred duringthe execution of the upgrade process, and the like.

At 308, the first host machine may determine a first notification levelassociated with the first notification generated in 306. For example,the first notification may include a flag or a field indicating thenotification level associated with the first notification.

At 310, the first host machine may compare the first notification levelto the user-specified notification level indicated in the configurationfile. At 312, the first host machine may then determine whether thefirst notification is to be sent to a user based on the comparingperformed in 310. At 314, the first host machine sends the firstnotification to the user if the first host machine determines in 312that the first notification is to be sent to the user, else thenotification is not sent to the user. The notification may be sent tothe user in various different forms such as an electronic alert, anemail messages, a text message, an SMS message, and the like.

Different techniques may be used in 310, 312, and 314 to determinewhether the first notification is to be sent to the user. One suchtechnique is depicted in FIG. 7. FIG. 7 depicts a simplified flowchart700 depicting a method performed by a host machine for determiningwhether a notification is to be sent to a user according to anembodiment of the present invention. The processing depicted in FIG. 7may be implemented in software (e.g., code, instructions, program)executed by one or more processing units (e.g., processors cores) of thehost machine, in hardware, or using combinations thereof. The softwaremay be stored on a non-transitory computer-readable medium (e.g., amemory device, a memory). The particular series of processing stepsdepicted in FIG. 7 is not intended to be limiting.

The flowchart 700 starts after the first host machine determines a firstnotification level associated with the first notification generated in306, at 308. For example, the first notification may include a flag or afield indicating the notification level associated with the firstnotification.

At 702, the first host machine reads a user-specified level from theconfiguration file. At 704, the first host machine determines whetherthe user-specified level is hierarchical. As provided above, two or moreof the levels may be organized in a hierarchy or ordering (e.g. aseverity or an importance hierarchy). For example, a hierarchy may bedefined for the PAUSEPOINT, ALERT, and INFO levels discussed above suchthat PAUSEPOINT corresponds to a higher hierarchical level than theALERT level, and the ALERT level corresponds to a higher hierarchicallevel than the INFO level. The hierarchical relationships between thelevels may influence the notifications that are sent to the users.

If the first host machine determines that the user-specified level ishierarchical (e.g., YES to 704), then, at 706, the first host machinedetermines whether the level determined in 308 is equal or higher thanthe user-specified level.

If the first host machine determines that the level determined in 308 isequal or higher than the user-specified level (e.g., YES to 706), thenthe first host machine sends the notification to the user at 710.

If the first host machine determines that the level determined in 308 isnot equal or higher than the user-specified level (e.g., NO to 706),then the first host machine does not send the notification to the userat 712.

On the other hand, returning back to 704, if the first host machinedetermines that the user-specified level is not hierarchical (e.g., NOto 704), then, at 708, the first host machine determines whether thelevel determined in 308 matches the user-specified level.

If the first host machine determines that the level determined in 308matches the user-specified level (e.g., YES to 708), then the first hostmachine sends the notification to the user at 710.

If the first host machine determines that the level determined in 308does not match the user-specified level (e.g., NO to 708), then thefirst host machine does not send the notification to the user at 712.

In some embodiments, the first host machine may be a part of a group ofhost machines (e.g., a pod) having a group configuration fileidentifying a user-specified group notification level. The groupnotification level may indicate a notification level that the user isinterested in for all hosts within the group. FIGS. 8A and 8B togetherdepict a simplified flowchart 800 for determining whether a notificationgenerated by a host machine is to be sent to a user when the hostmachine is part of a group of host machines (e.g., a pod) according toan embodiment of the present invention. The processing depicted in FIGS.8A and 8B may be implemented in software (e.g., code, instructions,program) executed by one or more processing units (e.g., processorscores) of a host machine, in hardware, or using combinations thereof.The software may be stored on a non-transitory computer-readable medium(e.g., a memory device, a memory). The particular series of processingsteps depicted in FIGS. 8A and 8B is not intended to be limiting.

At 802, a notification is generated on a first host machine. At 804, thefirst host machine determines a notification level for the generatednotification.

At 806, the first host machine determines whether the first host machineis part of a group of host machines (e.g., a pod) and a notificationlevel specified for the user in a configuration file for the group.

If the first host machine determines that the first host machine is partof a group of host machines (e.g., a pod) and a notification levelspecified for the user in a configuration file for the group (YES to806), then, at 808, the first host machine determines, based upon auser-specified notification level in the configuration file for thefirst host machine and the notification level determined in 804, whetherthe notification generated in 802 is to be sent to the user.

At 810, the first host machine sends the notification to the user if itis determined in 808 that the notification is to be sent to the user,else the first host machine does not send the notification to the user.

If the first host machine determines that the first host machine is notpart of a group of host machines (e.g., a pod) and a notification levelspecified for the user in a configuration file for the group (NO to806), then the processing moves to 812 in FIG. 8B. At 812, the firsthost machine determines whether there is a user-specified level inconfiguration file for the first host machine.

If there is a user-specified level in configuration file for the firsthost machine (YES to 812), then, at 814, the first host machinedetermines the level-to-use as the user-specified level in theconfiguration file.

If there is no user-specified level in configuration file for the firsthost machine (NO to 812), then, at 816, the first host machinedetermines the level-to-use as the user-specified level in the group(e.g., pod) configuration file.

At 818, the first host machine determines, based upon the level-to-useand the notification level determined in 804, whether the notificationgenerated in 802 is to be sent to the user.

At 820, the first host machine sends the notification to the user if itis determined in 818 that the notification is to be sent to the user,else the first host machine does not send the notification to the user.

As provided above, in certain embodiments, multiple notificationsgenerated by one or more host machines may be consolidated into a fewnumber of consolidated notifications. For example, multiplenotifications generated by one or more upgrade processes executing onone or more host machines may be consolidated into a single consolidatednotification and the consolidated notification then sent to a userinstead of the multiple notifications. FIG. 4 illustrates a simplifiedflowchart 400 depicting processing performed for consolidating multiplenotifications generated by one or more upgrade processes hosted by oneor more host machines according to an embodiment of the presentinvention. The processing depicted in FIG. 4 may be implemented insoftware (e.g., code, instructions, program) executed by one or moreprocessing units (e.g., processors cores), hardware, or combinationsthereof. The software may be stored in memory (e.g., on a memory device,on a non-transitory computer-readable storage medium). The particularseries of processing steps depicted in FIG. 4 is not intended to belimiting. In one embodiment, the processing depicted in FIG. 4 isperformed by one or more components of the upgrade infrastructure 200depicted in FIG. 2.

For the processing depicted in FIG. 4, it is assumed that a group ofmultiple host machines has been configured, the group including a firsthost machine and a second host machine. At 402, a first upgrade processis executed on the first host machine for upgrading a first softwareapplication on the first host machine. Similarly, at 404, a secondupgrade process may be executed on the second host machine for upgradinga second software application on the second host machine. The firstupgrade process and the second upgrade process may generatenotifications at various time points during their execution. Thenotifications for each upgrade process may include informationindicative of a status of that upgrade process when the notification isgenerated. For example, the notifications may indicate that the upgradeprocess has started execution, paused execution, is close to finishingexecution, or that a failure happened during the execution of theupgrade process, and the like.

For example, at 406, the first upgrade process may generate a firstnotification comprising information related to the first upgrade processexecuting on the first host machine. The first notification may indicatean upgrade progress status of the first upgrade process. At 408, thesecond upgrade process may generate a second notification comprisinginformation related to the second upgrade process executing on thesecond host machine. The second notification may indicate an upgradeprogress status of the second upgrade process.

Since the first and second host machines are part of a group of hostmachines (e.g., a pod), at 410, the first notification generated by thefirst host machine and the second notification generated by the secondhost machine are sent to a host machine within the group that is actingas the notification agent for that group of host machines. In someembodiments, the selection of a particular host machine within the groupas the notification agent may be done statically, for example,configured prior to the initiation of the upgrade operation. In such anembodiment, the notification agent for the group of host machinesremains the same and does not change. In some other embodiments, theselection of a particular host machine within the group as thenotification agent may be done dynamically and may change during theupgrade operation. In a dynamic embodiment, as part of 410, adetermination may first be made as to which host machine within thegroup of host machines is the notification agent and the first andsecond notifications are then sent to that notification agent hostmachine. It is possible that one of the first host machine or the secondhost machine acts as the notification agent.

In some embodiments, a dedicated host machine, possibly even outside thegroup of host machines, may be designated as the dedicated notificationagent as part of the upgrade infrastructure. The notifications are thensent to that notification agent.

At 412, the notification agent consolidates the first notification andthe second notification into a single consolidated notification. At 414,the consolidated notification is then sent to the user instead of thefirst and second notifications. The consolidated notification may besent to the user in various different forms such as an electronic alert,an email messages, a text message, an SMS message, and the like.

While consolidation of only two notifications is depicted in FIG. 4 anddescribed above, this is not meant to be restrictive. In general,multiple notifications may be consolidated in a fewer number ofconsolidated notifications, where the number of consolidatednotifications is lesser than the number of generated notifications. Themultiple notifications may be generated by the same upgrade process, bymultiple upgrade processes on the same host, or multiple upgradeprocesses on multiple hosts. The multiple notifications may beconsolidated into a fewer number of consolidated notifications or eveninto a single consolidate notification.

For example, according to one technique, information from the multiplenotifications may be collected and aggregated into the consolidatednotification. In another technique, the notification agent may generateinformation to be included in the consolidated notification based uponthe information contained in the multiple notifications to beconsolidated. For example, in one embodiment, the notification agent maygenerate summary information based the information contained in theindividual notifications to be consolidated and the summary informationmay be included in the consolidate notification that is generated andsent to the user.

In certain embodiments, the filtering of notifications based uponuser-specified levels and the consolidation of notifications may beperformed together by the notification agent. In certain embodiments,for a notification generated by a host machine, the host machine may beconfigured to first determine whether the notification is to be sent tothe notification agent based upon the notification level associated withthe generated notification and the user-specified level stored in theconfiguration file for the host machine. The host machine may only thesend the notification to the notification agent if the notificationlevel associated with the notification matches the user-specifiednotification level, or in the case of hierarchical levels, matches or ishigher than the user-specified notification level. In this manner, thenotification agent receives only a subset, and not all, of thenotifications generated by the host machines. The notification agent maythen form consolidated notifications based upon the receivednotifications. For example, for the scenarios depicted in FIG. 4 anddescribed above, the user may define a first configuration file for thefirst host machine and/or a second configuration file for the secondhost machine. The first host machine may decide whether or not to sendthe first notification to the notification agent based on the firstconfiguration file. The second host machine may decide whether or not tosend the second notification to the notification agent based on thesecond configuration file. Accordingly, the first configuration file andthe second configuration file may be applied to the first notificationand the second notification, respectively, prior to the notificationagent receiving the notifications.

In some other embodiments, the notification agent may be configured tofilter the received notifications based upon a user-specifiednotification level in the group-level configuration file (e.g., thepod-level configuration file). Accordingly, from the multiplenotifications received by the notification agent, the notification agentmay filter out some notifications based upon the group-levelconfiguration information. The notifications that are not filtered outmay then be consolidated by the notification agent and the consolidatednotifications then sent out to the user.

In yet another embodiment, notification levels may also be associatedwith the consolidate notifications and the notification agent maydetermine whether or not to send a consolidated notification to the userbased upon the level associated with the consolidated notification andthe user-specified level in the group-level configuration file for thatgroups of host machines. The notification agent may then only send outconsolidated notifications that pass the group-level based notificationlevel filter to the user.

As described above, embodiments provide various techniques for reducingthe number of notifications that are communicated to users from multiplehosts involved in an upgrade operation. These techniques includefiltering out notifications at the host machines based uponuser-specified level criteria, generating consolidated notifications,and combinations thereof. As a result, the number of notifications thatare actually sent to a user is far less than the number of notificationsthat are generated by the multiple upgrade processes executing on themultiple host machines. As a result, compared to conventionaltechniques, the number of network, processing, and memory resourcesneeded for processing of the notifications, both on the host machinesside and also on the client devices of users that receive thenotifications, are dramatically reduced.

FIG. 5 depicts a simplified diagram of a distributed system 500 forimplementing one of the embodiments. In the illustrated embodiment,distributed system 500 includes one or more client computing devices502, 504, 506, and 508, which are configured to execute and operate aclient application such as a web browser, proprietary client (e.g.,Oracle Forms), or the like over one or more network(s) 510. Server 512may be communicatively coupled with remote client computing devices 502,504, 506, and 508 via network 510.

In various embodiments, server 512 may be adapted to run one or moreservices or software applications provided by one or more of thecomponents of the system. In some embodiments, these services may beoffered as web-based or cloud services or under Software as a Service(SaaS) model to the users of client computing devices 502, 504, 506,and/or 508. Users operating client computing devices 502, 504, 506,and/or 508 may in turn utilize one or more client applications tointeract with server 512 to utilize the services provided by thesecomponents.

In the configuration depicted in the figure, the software components518, 520 and 522 of system 500 are shown as being implemented on server512. In other embodiments, one or more of the components of system 500and/or the services provided by these components may also be implementedby one or more of the client computing devices 502, 504, 506, and/or508. Users operating the client computing devices may then utilize oneor more client applications to use the services provided by thesecomponents. These components may be implemented in hardware, firmware,software, or combinations thereof. It should be appreciated that variousdifferent system configurations are possible, which may be differentfrom distributed system 500. The embodiment shown in the figure is thusone example of a distributed system for implementing an embodimentsystem and is not intended to be limiting.

Client computing devices 502, 504, 506, and/or 508 may be portablehandheld devices (e.g., an iPhone®, cellular telephone, an iPad®,computing tablet, a personal digital assistant (PDA)) or wearabledevices (e.g., a Google Glass® head mounted display), running softwaresuch as Microsoft Windows Mobile®, and/or a variety of mobile operatingsystems such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, andthe like, and being Internet, e-mail, short message service (SMS),Blackberry®, or other communication protocol enabled. The clientcomputing devices can be general purpose personal computers including,by way of example, personal computers and/or laptop computers runningvarious versions of Microsoft Windows®, Apple Macintosh®, and/or Linuxoperating systems. The client computing devices can be workstationcomputers running any of a variety of commercially-available UNIX® orUNIX-like operating systems, including without limitation the variety ofGNU/Linux operating systems, such as for example, Google Chrome OS.Alternatively, or in addition, client computing devices 502, 504, 506,and 508 may be any other electronic device, such as a thin-clientcomputer, an Internet-enabled gaming system (e.g., a Microsoft Xboxgaming console with or without a Kinect® gesture input device), and/or apersonal messaging device, capable of communicating over network(s) 510.

Although exemplary distributed system 500 is shown with four clientcomputing devices, any number of client computing devices may besupported. Other devices, such as devices with sensors, etc., mayinteract with server 512.

Network(s) 510 in distributed system 500 may be any type of networkfamiliar to those skilled in the art that can support datacommunications using any of a variety of commercially-availableprotocols, including without limitation TCP/IP (transmission controlprotocol/Internet protocol), SNA (systems network architecture), IPX(Internet packet exchange), AppleTalk, and the like. Merely by way ofexample, network(s) 510 can be a local area network (LAN), such as onebased on Ethernet, Token-Ring and/or the like. Network(s) 510 can be awide-area network and the Internet. It can include a virtual network,including without limitation a virtual private network (VPN), anintranet, an extranet, a public switched telephone network (PSTN), aninfra-red network, a wireless network (e.g., a network operating underany of the Institute of Electrical and Electronics (IEEE) 802.11 suiteof protocols, Bluetooth®, and/or any other wireless protocol); and/orany combination of these and/or other networks.

Server 512 may be composed of one or more general purpose computers,specialized server computers (including, by way of example, PC (personalcomputer) servers, UNIX® servers, mid-range servers, mainframecomputers, rack-mounted servers, etc.), server farms, server clusters,or any other appropriate arrangement and/or combination. In variousembodiments, server 512 may be adapted to run one or more services orsoftware applications described in the foregoing disclosure. Forexample, server 512 may correspond to a server for performing processingdescribed above according to an embodiment of the present disclosure.

Server 512 may run an operating system including any of those discussedabove, as well as any commercially available server operating system.Server 512 may also run any of a variety of additional serverapplications and/or mid-tier applications, including HTTP (hypertexttransport protocol) servers, FTP (file transfer protocol) servers, CGI(common gateway interface) servers, JAVA® servers, database servers, andthe like. Exemplary database servers include without limitation thosecommercially available from Oracle, Microsoft, Sybase, IBM(International Business Machines), and the like.

In some implementations, server 512 may include one or more applicationsto analyze and consolidate data feeds and/or event updates received fromusers of client computing devices 502, 504, 506, and 508. As an example,data feeds and/or event updates may include, but are not limited to,Twitter® feeds, Facebook® updates or real-time updates received from oneor more third party information sources and continuous data streams,which may include real-time events related to sensor data applications,financial tickers, network performance measuring tools (e.g., networkmonitoring and traffic management applications), clickstream analysistools, automobile traffic monitoring, and the like. Server 512 may alsoinclude one or more applications to display the data feeds and/orreal-time events via one or more display devices of client computingdevices 502, 504, 506, and 508.

Distributed system 500 may also include one or more databases 514 and516. Databases 514 and 516 may reside in a variety of locations. By wayof example, one or more of databases 514 and 516 may reside on anon-transitory storage medium local to (and/or resident in) server 512.Alternatively, databases 514 and 516 may be remote from server 512 andin communication with server 512 via a network-based or dedicatedconnection. In one set of embodiments, databases 514 and 516 may residein a storage-area network (SAN). Similarly, any necessary files forperforming the functions attributed to server 512 may be stored locallyon server 512 and/or remotely, as appropriate. In one set ofembodiments, databases 514 and 516 may include relational databases,such as databases provided by Oracle, that are adapted to store, update,and retrieve data in response to SQL-formatted commands.

FIG. 6 illustrates an exemplary computer system 600, in which variousembodiments of the present invention may be implemented. The system 600may be used to implement any of the computer systems described above. Asshown in the figure, computer system 600 includes a processing unit 604that communicates with a number of peripheral subsystems via a bussubsystem 602. These peripheral subsystems may include a processingacceleration unit 606, an I/O subsystem 608, a storage subsystem 618 anda communications subsystem 624. Storage subsystem 618 includes tangiblecomputer-readable storage media 622 and a system memory 610.

Bus subsystem 602 provides a mechanism for letting the variouscomponents and subsystems of computer system 600 communicate with eachother as intended. Although bus subsystem 602 is shown schematically asa single bus, alternative embodiments of the bus subsystem may utilizemultiple buses. Bus subsystem 602 may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. Forexample, such architectures may include an Industry StandardArchitecture (ISA) bus, Micro Channel Architecture (MCA) bus, EnhancedISA (EISA) bus, Video Electronics Standards Association (VESA) localbus, and Peripheral Component Interconnect (PCI) bus, which can beimplemented as a Mezzanine bus manufactured to the IEEE P1386.1standard.

Processing unit 604, which can be implemented as one or more integratedcircuits (e.g., a conventional microprocessor or microcontroller),controls the operation of computer system 600. One or more processorsmay be included in processing unit 604. These processors may includesingle core or multicore processors. In certain embodiments, processingunit 604 may be implemented as one or more independent processing units632 and/or 634 with single or multicore processors included in eachprocessing unit. In other embodiments, processing unit 604 may also beimplemented as a quad-core processing unit formed by integrating twodual-core processors into a single chip.

In various embodiments, processing unit 604 can execute a variety ofprograms in response to program code and can maintain multipleconcurrently executing programs or processes. At any given time, some orall of the program code to be executed can be resident in processor(s)604 and/or in storage subsystem 618. Through suitable programming,processor(s) 604 can provide various functionalities described above.Computer system 600 may additionally include a processing accelerationunit 606, which can include a digital signal processor (DSP), aspecial-purpose processor, and/or the like.

I/O subsystem 608 may include user interface input devices and userinterface output devices. User interface input devices may include akeyboard, pointing devices such as a mouse or trackball, a touchpad ortouch screen incorporated into a display, a scroll wheel, a click wheel,a dial, a button, a switch, a keypad, audio input devices with voicecommand recognition systems, microphones, and other types of inputdevices. User interface input devices may include, for example, motionsensing and/or gesture recognition devices such as the Microsoft Kinect®motion sensor that enables users to control and interact with an inputdevice, such as the Microsoft Xbox® 360 game controller, through anatural user interface using gestures and spoken commands. Userinterface input devices may also include eye gesture recognition devicessuch as the Google Glass® blink detector that detects eye activity(e.g., ‘blinking’ while taking pictures and/or making a menu selection)from users and transforms the eye gestures as input into an input device(e.g., Google Glass®). Additionally, user interface input devices mayinclude voice recognition sensing devices that enable users to interactwith voice recognition systems (e.g., Siri® navigator), through voicecommands.

User interface input devices may also include, without limitation, threedimensional (3D) mice, joysticks or pointing sticks, gamepads andgraphic tablets, and audio/visual devices such as speakers, digitalcameras, digital camcorders, portable media players, webcams, imagescanners, fingerprint scanners, barcode reader 3D scanners, 3D printers,laser rangefinders, and eye gaze tracking devices. Additionally, userinterface input devices may include, for example, medical imaging inputdevices such as computed tomography, magnetic resonance imaging,position emission tomography, medical ultrasonography devices. Userinterface input devices may also include, for example, audio inputdevices such as MIDI keyboards, digital musical instruments and thelike.

User interface output devices may include a display subsystem, indicatorlights, or non-visual displays such as audio output devices, etc. Thedisplay subsystem may be a cathode ray tube (CRT), a flat-panel device,such as that using a liquid crystal display (LCD) or plasma display, aprojection device, a touch screen, and the like. In general, use of theterm “output device” is intended to include all possible types ofdevices and mechanisms for outputting information from computer system600 to a user or other computer. For example, user interface outputdevices may include, without limitation, a variety of display devicesthat visually convey text, graphics and audio/video information such asmonitors, printers, speakers, headphones, automotive navigation systems,plotters, voice output devices, and modems.

Computer system 600 may comprise a storage subsystem 618 that comprisessoftware elements, shown as being currently located within a systemmemory 610. System memory 610 may store program instructions that areloadable and executable on processing unit 604, as well as datagenerated during the execution of these programs.

Depending on the configuration and type of computer system 600, systemmemory 610 may be volatile (such as random access memory (RAM)) and/ornon-volatile (such as read-only memory (ROM), flash memory, etc.) TheRAM typically contains data and/or program modules that are immediatelyaccessible to and/or presently being operated and executed by processingunit 604. In some implementations, system memory 610 may includemultiple different types of memory, such as static random access memory(SRAM) or dynamic random access memory (DRAM). In some implementations,a basic input/output system (BIOS), containing the basic routines thathelp to transfer information between elements within computer system600, such as during start-up, may typically be stored in the ROM. By wayof example, and not limitation, system memory 610 also illustratesapplication programs 612, which may include client applications, Webbrowsers, mid-tier applications, relational database management systems(RDBMS), etc., program data 614, and an operating system 616. By way ofexample, operating system 616 may include various versions of MicrosoftWindows®, Apple Macintosh®, and/or Linux operating systems, a variety ofcommercially-available UNIX® or UNIX-like operating systems (includingwithout limitation the variety of GNU/Linux operating systems, theGoogle Chrome® OS, and the like) and/or mobile operating systems such asiOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OSoperating systems.

Storage subsystem 618 may also provide a tangible computer-readablestorage medium for storing the basic programming and data constructsthat provide the functionality of some embodiments. Software (programs,code modules, instructions) that when executed by a processor providethe functionality described above may be stored in storage subsystem618. These software modules or instructions may be executed byprocessing unit 604. Storage subsystem 618 may also provide a repositoryfor storing data used in accordance with the present invention.

Storage subsystem 600 may also include a computer-readable storage mediareader 620 that can further be connected to computer-readable storagemedia 622. Together and, optionally, in combination with system memory610, computer-readable storage media 622 may comprehensively representremote, local, fixed, and/or removable storage devices plus storagemedia for temporarily and/or more permanently containing, storing,transmitting, and retrieving computer-readable information.

Computer-readable storage media 622 containing code, or portions ofcode, can also include any appropriate media known or used in the art,including storage media and communication media, such as but not limitedto, volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information. This can include tangible computer-readable storagemedia such as RAM, ROM, electronically erasable programmable ROM(EEPROM), flash memory or other memory technology, CD-ROM, digitalversatile disk (DVD), or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or other tangible computer readable media. This can also includenontangible computer-readable media, such as data signals, datatransmissions, or any other medium which can be used to transmit thedesired information and which can be accessed by computing system 600.

By way of example, computer-readable storage media 622 may include ahard disk drive that reads from or writes to non-removable, nonvolatilemagnetic media, a magnetic disk drive that reads from or writes to aremovable, nonvolatile magnetic disk, and an optical disk drive thatreads from or writes to a removable, nonvolatile optical disk such as aCD ROM, DVD, and Blu-Ray® disk, or other optical media.Computer-readable storage media 622 may include, but is not limited to,Zip® drives, flash memory cards, universal serial bus (USB) flashdrives, secure digital (SD) cards, DVD disks, digital video tape, andthe like. Computer-readable storage media 622 may also include,solid-state drives (SSD) based on non-volatile memory such asflash-memory based SSDs, enterprise flash drives, solid state ROM, andthe like, SSDs based on volatile memory such as solid state RAM, dynamicRAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, andhybrid SSDs that use a combination of DRAM and flash memory based SSDs.The disk drives and their associated computer-readable media may providenon-volatile storage of computer-readable instructions, data structures,program modules, and other data for computer system 600.

Communications subsystem 624 provides an interface to other computersystems and networks. Communications subsystem 624 serves as aninterface for receiving data from and transmitting data to other systemsfrom computer system 600. For example, communications subsystem 624 mayenable computer system 600 to connect to one or more devices via theInternet. In some embodiments communications subsystem 624 can includeradio frequency (RF) transceiver components for accessing wireless voiceand/or data networks (e.g., using cellular telephone technology,advanced data network technology, such as 3G, 4G or EDGE (enhanced datarates for global evolution), WiFi (IEEE 802.11 family standards, orother mobile communication technologies, or any combination thereof),global positioning system (GPS) receiver components, and/or othercomponents. In some embodiments communications subsystem 624 can providewired network connectivity (e.g., Ethernet) in addition to or instead ofa wireless interface.

In some embodiments, communications subsystem 624 may also receive inputcommunication in the form of structured and/or unstructured data feeds626, event streams 628, event updates 630, and the like on behalf of oneor more users who may use computer system 600.

By way of example, communications subsystem 624 may be configured toreceive data feeds 626 in real-time from users of social networks and/orother communication services such as Twitter® feeds, Facebook® updates,web feeds such as Rich Site Summary (RSS) feeds, and/or real-timeupdates from one or more third party information sources.

Additionally, communications subsystem 624 may also be configured toreceive data in the form of continuous data streams, which may includeevent streams 628 of real-time events and/or event updates 630, that maybe continuous or unbounded in nature with no explicit end. Examples ofapplications that generate continuous data may include, for example,sensor data applications, financial tickers, network performancemeasuring tools (e.g. network monitoring and traffic managementapplications), clickstream analysis tools, automobile trafficmonitoring, and the like.

Communications subsystem 624 may also be configured to output thestructured and/or unstructured data feeds 626, event streams 628, eventupdates 630, and the like to one or more databases that may be incommunication with one or more streaming data source computers coupledto computer system 600.

Computer system 600 can be one of various types, including a handheldportable device (e.g., an iPhone® cellular phone, an iPad® computingtablet, a PDA), a wearable device (e.g., a Google Glass® head mounteddisplay), a PC, a workstation, a mainframe, a kiosk, a server rack, orany other data processing system.

Due to the ever-changing nature of computers and networks, thedescription of computer system 600 depicted in the figure is intendedonly as a specific example. Many other configurations having more orfewer components than the system depicted in the figure are possible.For example, customized hardware might also be used and/or particularelements might be implemented in hardware, firmware, software (includingapplets), or a combination. Further, connection to other computingdevices, such as network input/output devices, may be employed. Based onthe disclosure and teachings provided herein, a person of ordinary skillin the art will appreciate other ways and/or methods to implement thevarious embodiments.

In the foregoing specification, aspects of the invention are describedwith reference to specific embodiments thereof, but those skilled in theart will recognize that the invention is not limited thereto. Variousfeatures and aspects of the above-described invention may be usedindividually or jointly. Further, embodiments can be utilized in anynumber of environments and applications beyond those described hereinwithout departing from the broader spirit and scope of thespecification. The specification and drawings are, accordingly, to beregarded as illustrative rather than restrictive.

What is claimed is:
 1. A method comprising: executing, on each hostmachine of a plurality a host machine, at least one upgrade process froma plurality of upgrade processes; determining, from among the pluralityof host machines, a particular host machine as a notification agent forthe plurality of host machines, wherein the particular host machineexecutes a third upgrade process of the plurality of upgrade processes;receiving, by the particular host machine from the plurality of hostmachines, a first notification, the first notification generated by afirst upgrade process of the plurality of upgrade processes executed bya first host machine from the plurality of host machines, the firstupgrade process for upgrading a first software application on the firsthost machine, the first notification comprising information related tothe first upgrade process; generating, by the particular host machine, asingle notification by consolidating the first notification and a secondnotification, the second notification generated by a second upgradeprocess of the plurality of upgrade processes executed by a second hostmachine from the plurality of host machines, the second upgrade processfor upgrading a second software application on the second host machine,the second notification comprising information related to the secondupgrade process; sending, by the particular host machine, the singlenotification to a user instead of the first notification and the secondnotification; and subsequent to sending the single notification,selecting, from among the plurality of host machines, a differentparticular host machine other than the particular host machine, as thenotification agent for the plurality of the host machines.
 2. The methodof claim 1, wherein the first notification is generated before thesecond notification.
 3. The method of claim 1, wherein the plurality ofupgrade processes are part of an upgrade operation, and the methodfurther comprising: maintaining the particular host machine as thenotification agent during the upgrade operation.
 4. The method of claim1, wherein the plurality of upgrade processes are part of an upgradeoperation, and the method further comprising: changing the notificationagent during the upgrade operation.
 5. The method of claim 1, whereinthe determining is based upon: a type of notification of the firstnotification or the second notification; or a start time of the firstupgrade process, the second upgrade process, or the third upgradeprocess.
 6. The method of claim 1, further comprising: determiningwhether to send the single notification to the user, wherein thedetermining is based upon a notification level of the singlenotification and a user-specified level for the plurality of hostmachines, wherein the single notification is sent to the user inresponse to determining to send the single notification to the user. 7.The method of claim 1, wherein the single notification is sent to theuser after a predefined delay after an upgrade process of the particularhost machine begins.
 8. A non-transitory computer-readable storagemedium storing a plurality of instructions executable by one or moreprocessors, the plurality of instructions when executed by the one ormore processors causes the one or more processors to: execute, by aparticular host machine of a plurality of host machines, a third upgradeprocess from a plurality of upgrade processes, wherein each host machineof the plurality of host machines executes at least one upgrade processfrom the plurality of upgrade processes; determine, the particular hostmachine of the plurality of host machines, as a notification agent forthe plurality of host machines; receive, by the particular host machine,a first notification, the first notification generated by a firstupgrade process of the plurality of upgrade processes executed by afirst host machine, the first upgrade process for upgrading a firstsoftware application on the first host machine, the first notificationcomprising information related to the first upgrade process; generate,by the particular host machine, a single notification by consolidatingthe first notification and a second notification of the plurality ofupgrade processes, the second notification generated by a second upgradeprocess executed by a second host machine, the second upgrade processfor upgrading a second software application on the second host machine,the second notification comprising information related to the secondupgrade process; send the single notification to a user instead of thefirst notification and the second notification; and subsequent tosending the single notification, select, from among the plurality ofhost machines, a different particular host machine other than theparticular host machine, as the notification agent for the plurality ofthe host machines.
 9. The non-transitory computer-readable storagemedium of claim 8, wherein the first notification is generated beforethe second notification.
 10. The non-transitory computer-readablestorage medium of claim 8, wherein the plurality of upgrade processesare part of an upgrade operation, and the plurality of instructions whenexecuted by the one or more processors further causes the one or moreprocessors to: maintain the particular host machine as the notificationagent during the upgrade operation.
 11. The non-transitorycomputer-readable storage medium of claim 8, wherein the plurality ofupgrade processes are part of an upgrade operation, and the plurality ofinstructions when executed by the one or more processors further causesthe one or more processors to: change the notification agent during theupgrade operation.
 12. The non-transitory computer-readable storagemedium of claim 8, the plurality of instructions when executed by theone or more processors further causes the one or more processors to:subsequent to sending the single notification, reassigning a differenthost machine as the notification agent.
 13. The non-transitorycomputer-readable storage medium of claim 8, wherein the determining isbased upon one or more of the following: a type of notification of thefirst notification or the second notification; or a start time of thefirst upgrade process, the second upgrade process, or the third upgradeprocess.
 14. The non-transitory computer-readable storage medium ofclaim 8, wherein the plurality of instructions when executed by the oneor more processors further causes the one or more processors to:determine whether to send the single notification to the user, whereinthe determining is based upon a notification level of the singlenotification and a user-specified level for a plurality of host machinesincluding the first host machine and the second host machine, whereinthe single notification is sent to the user in response to determiningto send the single notification to the user.
 15. The non-transitorycomputer-readable storage medium of claim 8, wherein the singlenotification is sent to the user after a predefined delay after thethird upgrade process begins.
 16. A system comprising: one or moreprocessors; and a non-transitory computer-readable medium includinginstructions that, when executed by the one or more processors, causethe one or more processors to: execute, by a particular host machine ofa plurality of host machines, a third upgrade process from a pluralityof upgrade processes, wherein each host machine of the plurality of hostmachines executes at least one upgrade process from the plurality ofupgrade processes; determine, the particular host machine of theplurality of host machines, as a notification agent for the plurality ofhost machines; receive a first notification, the first notificationgenerated by a first upgrade process of the plurality of upgradeprocesses executed by a first host machine, the first upgrade processfor upgrading a first software application on the first host machine,the first notification comprising information related to the firstupgrade process; generate a single notification by consolidating thefirst notification and a second notification of the plurality of upgradeprocesses, the second notification generated by a second upgrade processexecuted by a second host machine, the second upgrade process forupgrading a second software application on the second host machine, thesecond notification comprising information related to the second upgradeprocess; send the single notification to a user instead of the firstnotification and the second notification; and subsequent to sending thesingle notification, select, from among the plurality of host machines,a different particular host machine other than the particular hostmachine, as the notification agent for the plurality of the hostmachines.
 17. The system of claim 16, wherein the instructions, whenexecuted by the one or more processors, further cause the one or moreprocessors to: determine whether to send the single notification to theuser, wherein the determining is based upon a notification level of thesingle notification and a user-specified level for a plurality of hostmachines including the first host machine and the second host machine,wherein the single notification is sent to the user in response todetermining to send the single notification to the user.